Business Data Analytics Challenge:

Prediction of Electricity Flows

Fabio Costa
Laurin Eichberger
Nevena Nikolajevic
Ömer Özumerzifon
Lukas Petry
Hendrik Scherner

In [4]:
# Disable all warnings in this notebook to improve readability. 
import warnings
warnings.simplefilter("ignore")

1 Domain Understanding

Predict the cross-border power flow between Germany and its neighbors

European integration of electricity market is triggering…

  • Need to predict the physical net current flow of Germany to the neighboring countries
  • Adequate reactions facilitated by reliable predictions

Goal of this project is to develop a ML-based model that can reliably perform the predictions based on publicly available data input.

Predict the cross-border power flow between Germany and its neighbors

Output requirements of the model:

  • Time stamp with hourly granularity
  • Net electricity flow per country

The model is not trained again before the input of future data and shall therefore be generalizable.

Quality indicator of the prediction is quantity-weighted RSME.

Possible Influences

First step was a brainstorming about the possible influence factors without specifying potential data sources and required formats. Afterwards we clustered the factors into a mind-map.

The main factor categories we identified are:

  • Market-related data
  • Physical restrictions
  • Production and Weather data

Besides data related to Germany, the influence factors can always be extended to neighboring countries, this means that most of the identified factors exist for every other country and could thus be used as further input for training our model.

However, our main independent variables are related to Germany, improved by weather data from neighboring countries.

Possible Influences

Possible Influences

In [6]:
%%html
<style>
  table {margin-left: 0 !important;}
</style>

Data sources

Only publicly availabe data:


Extracted Data Source
Net Electricity Flow SMARD, xyz
Production forecast SMARD, xyz
Consumption forecast SMARD, xyz
Realized Prodcution SMARD, xyz
Realized consumption SMARD, xyz
Price Data SMARD, xyz
Weather DWD, DMI,
LFBD, LKPR


  • SMARD, DWD and DMI data was availale for June 2015 - June 2019

  • Rest?

  • We built a scraper for the German and Danish weather data (DWD, DMI)

Abbreviations:
DWD (German Meteorological Institute)
DMI (Danish Meteorological Institute)
LFBD (Airport Bordeaux)
LKPR (Airport Prague)

Fazit

Oft ist es gut, wenn am Ende eines Kapitels, bzw. vor dem Anfang des nächsten Kapitels noch ein Absatz steht, der alles zusammenfasst. Bei 10 Minuten egal? Was meint ihr?

2 Discover and visualize the data to gain insight

  • Bokeh
  • Korrelationsplots von Hendrik
  • Prophet Decomposition
In [8]:
from bda_bokeh import bda_bokeh
df = bda_bokeh.import_data()
df = bda_bokeh.preprocessing(df)
bda_bokeh.show_bokeh(df)

Import File: DE_Physikalischer Stromfluss_201506010000_201706012359_1.csv 
Import File: DE_Physikalischer Stromfluss_201706020000_201906022359_1.csv
Loading BokehJS ...

@TODO Korrelationsplots von Hendrik

In [9]:
from bda_prophet import bda_prophet
from fbprophet import Prophet

#from fbprophet.plot import plot_plotly
#import plotly.offline as py

country_codes = bda_prophet.extract_country_codes(df)
df = bda_prophet.get_net_export(df, country_codes, ["EX","IM"])
df.head()

df_NX = df[['Date', 'NX']]
df_NX = bda_prophet.rename_columns_for_prophet(df_NX)
df_NX.head()

m = Prophet()
m.fit(df_NX)

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-9-550ddd5a9981> in <module>()
----> 1 from bda_prophet import bda_prophet
      2 from fbprophet import Prophet
      3 
      4 #from fbprophet.plot import plot_plotly
      5 #import plotly.offline as py

~\Documents\GitHub\BDA-Presentation\bda_prophet\bda_prophet.py in <module>()
      6 from datetime import timedelta
      7 from datetime import datetime
----> 8 from fbprophet import Prophet
      9 
     10 def extract_country_codes(df):

ModuleNotFoundError: No module named 'fbprophet'

A decomposition of linear components with Facebook's Prophet creates additional insights.

In [42]:
import pandas as pd
from datetime import datetime

future = pd.DataFrame({"ds": pd.date_range(start=datetime(2018,5,1), end=datetime(2019,5,31,23,0,0))})
forecast = m.predict(future) 
fig2 = m.plot_components(forecast)

3 General Preprocessing

Zeigen, wie wir auf den Masterdataframe gekommen sind.

4 Selecting a Model

Possible Models

Überblick geben mit der Liste am Anfang

Contestants

Näher infrage kommende Modelle

Final selection

Erklären, welche(s) Modell(e) wir am Ende genommen haben, und warum.

5 Explaining the model

Erklären wie das finale Modell funktioniert, Input, Output, Funktionsweise ...

6 Results

Presentation

Einfach Plots zeigen.

Interpretation

Erklären, wo das Modell stark und wo es schwach ist.

7 Conclusion

Was haben wir gelernt, wo ist Verbesserungpotential.

In [ ]: